A Nonlinear Noise-Shaping Delta-Sigma Modulator with On-Chip Reinforcement Learning∗

The stability and quality of noise shaping is a concern in the design of higher-order delta-sigma modulators for oversampled analog-to-digital conversion. We reformulate noise-shaping modulation alternatively as a nonlinear control problem, where the objective is to find the binary modulation sequence that minimizes signal swing in a cascade of integrators operating on the difference between the input signal and the modulation sequence. Reinforcement learning is used to adaptively optimize a nonlinear neural classifier, which outputs modulation bits from the values of the input signal and integration state variables. Analogous to the pole balancing control problem, a punishment signal triggers learning whenever any of the integrators saturate. Experimental results obtained from a VLSI modulator with integrated classifier, trained to produce stable noise shaping modulation of orders one and two, are presented. The classifier contains an array of 64 locally tuned, binary address-encoded neurons and is trained on-chip with a variant on reinforcement learning.